System for output of dual video stream via a single parallel digital video interface

ABSTRACT

A method of operating a video camera includes capturing a scene of imaging data using a focal plane array (FPA) module of the video camera. The scene of imaging data is characterized by a first bit depth. The method also includes processing, using an image processing module coupled to the FPA module, the scene of imaging data to provide display data characterized by a second bit depth less than the first bit depth. The method further includes forming a super frame including the display data and the scene of imaging data and outputting the super frame.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/439,831, filed on Feb. 22, 2017, which is a divisional of U.S. patentapplication Ser. No. 14/536,439, filed on Nov. 7, 2014, now U.S. Pat.No. 9,615,037, which claims priority to U.S. Provisional PatentApplication No. 61/901,817, filed on Nov. 8, 2013, entitled “Method andSystem for Output of Dual Video Stream via a Single Parallel DigitalVideo Interface,” the disclosures of which are hereby incorporated byreference in their entirety for all purposes.

SUMMARY OF THE INVENTION

The present invention relates generally to video systems. Merely by wayof example, the methods, systems, and apparatuses described herein havebeen applied to video processing and delivery of video streams in videocameras, including thermal imaging cameras. The invention has wideapplicability to video data and the delivery thereof.

Although video interfaces have been demonstrated, embodiments of thepresent invention provide functionality not available using conventionaltechniques. As described herein, a single interface is utilized toprovide two video streams that contain different levels of videoprocessing on the same sensor content. In other words, a single videostream is provided by embodiments that contains video imagery with twodifferent levels of video processing from the same sensor.

According to an embodiment of the present invention, a method ofoperating a video camera is provided. The method includes capturing ascene of imaging data using the video camera, wherein the imaging datais characterized by a first bit depth and processing the imaging data toprovide display data characterized by a second bit depth less than thefirst bit depth. The method also includes framing the imaging data andthe display data and outputting the framed imaging and display data.

According to another embodiment of the present invention, a method ofoperating a thermal imaging system is provided. The method includescapturing a scene of imaging data using a thermal imager, wherein theimaging data is characterized by a first bit depth and processing theimaging data to provide display data characterized by a second bit depthless than the first bit depth. The method also includes processing theimaging data to provide radiometric data characterized by a third bitdepth greater than the first bit depth and framing the radiometric dataand the display data. The method further includes outputting the framedradiometric and display data.

According to a specific embodiment of the present invention, a thermalimaging system is provided. The thermal imaging system includes one ormore optical elements operable to collect infrared light and a cameracore optically coupled to the one or more optical elements. The cameracore includes an FPA module providing imaging data at a first bit depth,a color conversion module coupled to the FPA module and operable toprocess the imaging data to provide display data, and a framer coupledto the FPA module and the color conversion module and operable to form asuper frame including the imaging data and the display data. The thermalimaging system also includes a communications module coupled to thecamera core and an input/output module coupled to the communicationsmodule.

Numerous benefits are achieved by way of these techniques overconventional methods. For example, embodiments provide a method tooutput two video streams from a camera core in a single super frame orvideo stream without adding any significant cost, power, or cabling/pinsonto the core. Some applications include processing of one of the twostreams through a video analytics algorithm while the other of the twostreams is used as the “viewable” (e.g., human viewable) stream. Usingembodiments of the present invention, a single video stream thatcontains video imagery with two different levels of video processing isprovided. This contrasts with conventional methods in which two videostreams in the form of two separate packets of data with two differentdestination ports are used to communicate signals with different levelsof processing. In embodiments, of the present invention, a single streamis used that is output at a single destination port. These and otherdetails of embodiments along with many of their advantages and featuresare described in the following description, claims, and figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified block diagram of a dual stream video systemaccording to an embodiment of the present invention.

FIG. 2 is a simplified block diagram of a dual stream video systemaccording to an alternative embodiment of the present invention.

FIG. 3 is diagram illustrating a frame format for 320 pixel wide videooutput according to an embodiment of the present invention.

FIG. 4 is diagram illustrating a frame format for 640 pixel wide videooutput according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating a super frame according to anembodiment of the present invention.

FIG. 6A is a simplified flowchart illustrating a method of providing adual video stream according to an embodiment of the present invention.

FIG. 6B is a simplified flowchart illustrating a method of providing adual video stream using radiometry data according to an embodiment ofthe present invention.

FIG. 7 is an LVDS timing diagram according to an embodiment of thepresent invention.

FIG. 8 is an LVDS window timing diagram according to an embodiment ofthe present invention.

FIG. 9 illustrates the mapping of a Camera Link® serialized bit streamto a 24 bit RGB color bit stream according to an embodiment of thepresent invention.

FIG. 10 illustrates a YUV super frame line format for one line of dataaccording to an embodiment of the present invention.

FIG. 11 is a simplified block diagram illustrating a thermal imagingsystem according to an embodiment of the present invention.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Embodiments of the present invention provide methods and systems tooutput a ‘super frame’ that includes both ‘raw’ video (e.g., 14-bit‘raw’ video, which can be utilized by an analytics engine) as well ascontrast enhanced video (e.g., 8-bit contrast enhanced video suitablefor use with imaging displays) from a thermal camera core using asingle, parallel digital video interface. In other embodiments asdescribed herein, the Super Frame can include 16-bit radiometric dataalong with 8-bit gray scale data represented, for example, in YUVformat. Embodiments of the present invention utilize a ‘super frame’format to provide both video streams in which the number of pixels perline can be doubled. Thus, embodiments of the present invention cantransmit two different images or representations of the same scene in asingle frame. In other words, the method provides a clean solution tooutput two (or more) representations of the same sensor output data thathave different levels of video processing applied.

Thus, embodiments of the present invention utilize a camera core thatoutputs two video streams of different bit depth resolution and providethem, for example, to a video board that outputs an Ethernet IP stream.The sensor data obtained by the video core is processed at multiplelevels in the camera core as described herein to provide the twodifferent video streams that are based on the same sensor data. Incontrast with conventional techniques, the super frame is utilized toenable a single physical interface to carry two video streams providedby the processing engine in the camera core.

FIG. 1 is a simplified block diagram of a dual stream video systemaccording to an embodiment of the present invention. The dual streamvideo system 100, which can be referred to as a camera core, includes afocal plane array (FPA) 110 that collects incident radiation. As anexample, the FPA 110 can be a 17 μm pixel pitch, long wave infrared(LWIR) FPA that is useful in obtaining thermal video and/or stillimagery. The FPA, also referred to as a sensor, can have one of severalresolutions, including 640×480, 320×240, or the like. Of course, otherpixel pitch, operating wavelengths, and sensor resolutions are includedwithin the scope of the present invention. In the illustratedembodiment, the bit depth of the signal output by the FPA is 14 bits.This 14-bit raw data stream is used in an exemplary manner throughoutthis disclosure, but other bit depths are included within the scope ofthe present invention. One of ordinary skill in the art would recognizemany variations, modifications, and alternatives.

The video signal from the FPA 110 is provided to a non-uniformitycorrection (NUC) module 112 that corrects for pixel-to-pixelnon-uniformities present across the sensor as well as performingoptional temperature compensation. The NUC module 112 provides a 14-bitnon-uniformity corrected output to multiple modules in the embodimentillustrated in FIG. 1. The FPA 110 and the NUC module 112 can beconsidered as an image capture module that receives infrared light,corrects for detector non-uniformity, and outputs imaging data. In someembodiments, the imaging data is referred to as raw data, but it shouldbe understood that the use of the term raw data in this context includesprocessing such as non-uniformity correction and other suitable imagingprocessing techniques.

FIG. 1 illustrates the multiple video processing paths that are utilizedto implement the embodiments described herein. Additional descriptionrelated to non-uniformity correction is provided in U.S. patentapplication Ser. No. 14/211,796, filed on Mar. 14, 2014, the disclosureof which is hereby incorporated by reference in its entirety for allpurposes. The output of the NUC module 112 is referred to herein as‘raw’ data, signifying that the data has not been processed other thanfor the non-uniformity correction. Embodiments of the present inventionare not limited to ‘raw’ data in the sense of only non-uniformitycorrection, but include implementations in which data processed at afirst bit depth is provided in the super frame interleaved with data ofa second, reduced bit depth.

If the raw 14-bit image after NUC was displayed to a user, the qualitywould be very poor since the image would be gray and washed out.Accordingly, to improve the display experience, the raw data is providedat the original bit depth to the automatic gain control (AGC)/local areaprocessing (LAP) module 120 in which AGC and/or LAP is performed. Thismodule can also be referred to as an image contrast enhancement module.The AGC/LAP module 120 performs contrast enhancement processing, edgedetection, and the like to provide 8-bit video imagery that is suitablefor use in user displays. This 8-bit video stream can be displayed tousers, providing the desired contrast.

Although 8-bit data is illustrated in FIG. 1, this is not required bythe present invention and other reduced bit depths can be utilized asappropriate to the particular application. The contrast enhanced videodata output from the AGC/LAP module 120 is provided to the multiplexer140, which can select this contrast enhanced video data as an output tothe parallel interface 150.

As illustrated in FIG. 1, the raw data from the NUC module 112 and thecontrast enhanced data (i.e., the 8-bit video stream) is provided to acolorizer module 122 that converts the gray scale/contrast enhanced datato a color format, also referred to as a color video stream, forexample, the 4:2:2 color format, the YUV color format, or the like. Thecolorizer module 122 can also be referred to as a YUV processing module.In the 4:2:2 color format, the gray scale or luminance information ofthe pixel (4 bytes of Y data) is provided at twice the data size of thecolor information (2 bytes of U (Cb) data and 2 bytes of V (Cr) data),providing a 16-bit color (i.e., 4:2:2) video signal. The AGC/LAP module120 and the colorizer module 122 can be considered as a color conversionmodule that receives imaging data and outputs display data.

In one implementation, the 4:2:2 data can be treated as an array ofunsigned char values, where the first byte contains the first Y sample,the second byte contains the first U (Cb) sample, the third bytecontains the second Y sample, and the fourth byte contains the first V(Cr) sample, as shown in Table 1, with increasing memory addressesproceeding to the right.

TABLE 1 Y0 U0 Y1 V1 Y2 U1 Y3 V1 Y4 U2 Y5 V2If the image is addressed as an array of little-endian WORD values, thefirst WORD contains the first Y sample in the least significant bits(LSBs) and the first U (Cb) sample in the most significant bits (MSBs).The second WORD contains the second Y sample in the LSBs and the first V(Cr) sample in the MSBs.

The output of the colorizer module 122 is a 16-bit per pixel color videostream that is provided to the framer 130 for framing into the superframe. In other embodiments, other colorization protocols can be used toprovide colorized data for framing and eventual display. One of ordinaryskill in the art would recognize many variations, modifications, andalternatives.

Because some processing devices, for example, video analytics systems,can benefit from receiving the 14-bit video stream rather than the 8-bitvideo stream, embodiments of the present invention provide a dual videostream output from the camera core as described herein that is suitablefor both processing of the 14-bit video stream as well as for display.Referring to FIG. 1, the 14-bit raw data is provided to framer 130 atthe input 141. The framer 130 frames the 14-bit video signal with the16-bit color video signal into a super frame as described more fullybelow.

The output of the framer 130 is provided to multiplexer 140, which canselect one of the inputs to be provided as an output to the parallelinterface 150. The framer is able to buffer up to a line of data in oneimplementation and more than one line of data in other implementations.

Although FIG. 1 illustrates a particular example of framing the 14-bitraw video stream with the 16-bit color video stream, embodiments of thepresent invention are not limited to this particular implementation. Itis possible to frame other video streams provided at different points ofthe processing flow, for example, 8-bit gray scale and 14-bit raw dataafter NUC, 16-bit color and 14-bit raw data, 8-bit gray scale and 16-bitradiometric as described below, and the like.

The multiplexer 140 receives a plurality of inputs, for example, the raw14-bit video data (input 145), the contrast enhanced 8-bit video data(input 142), the colorized 16-bit video data (input 143), and the superframe video data (input 144) and selects one of these inputs as theoutput that is provided to the parallel interface 150 based on the inputprovided at the select line SEL. Although framing of the raw data andthe colorized data into a super frame is illustrated in FIG. 1, otherembodiments combine the raw data and the contrast enhanced data in asuper frame by providing the output of the AGC/LAP module 120 to theframer 130 instead of the output of the colorizer module 122.Accordingly, the multiplexer can select either one of the video streams(e.g., the display video stream, the colorized stream, the raw videostream, or the super frame video stream for delivery to the parallelinterface 150 or to the LVDS interface discussed below.

FIG. 2 is a simplified block diagram of a dual stream video system 200according to an alternative embodiment of the present invention. In theembodiment illustrated in FIG. 2, which shares similarities with thesystem illustrated in FIG. 1, the 14-bit raw data is converted to 16-bitradiometric data by radiometric module 210. The description provided inrelation to the elements in FIG. 1 is applicable to the elementsillustrated in FIG. 2 as appropriate.

The radiometry module 210 converts the 14-bit intensity data into a16-bit temperature value, represented by 11 bits to define the integerportion of the temperature value in Kelvin and 5 bits to define thefractional portion of the temperature value in Kelvin. Of course, otherconfigurations can be utilized in which fewer or more bits are used todefine the integer and fractional portions of the value. The temperaturedata is provided to the multiplexer as input 212 and is framed togetherwith either the colorized data (input 143) or the contrast enhanced data(not illustrated but available by providing input 142 to the framer 130)to form the super frame featuring radiometric temperature information.The radiometry module 210 can utilize radiometric lookup tables or otherradiometric processing to convert the 14-bit video stream to produce a16-bit radiometric video stream using the camera temperature,calibration data, and the like.

The framer 130 combines the 16-bit radiometric video stream with one ofthe other video signals (e.g., the contrast enhanced 8-bit video data(input 142) or the colorized 16-bit video data (input 143) for form thesuper frame. As illustrated in FIG. 2, the radiometric video data (input245) can be passed through the multiplexer as an input to the parallelinterface 150. In some embodiments, the raw data can also be passedthrough the multiplexer as an input to the parallel interface asillustrated by input 247 to multiplexer 140.

FIG. 3 is diagram illustrating a frame format for 320 pixel wide videooutput according to an embodiment of the present invention. The 4:2:2super frame format shown in FIG. 3 is for a 320×240 sensor with 240 rowsand illustrates how each row of 4:2:2 data is transmitted followed bythe corresponding row of raw data. For two bytes per column, there are640 bytes in each subframe. Accordingly, since the data is essentially2-bytes per pixel, two clock cycles are used to clock out the data insome implementations.

Referring to FIG. 3, in the super frame, the first 640 bytes are for theprocessed data (e.g., color YUV data), with 16 bits per pixel. Thesecond 640 bytes are the raw data (e.g., 16 bit radiometric or 14 bitraw data). Accordingly, each subframe of the super frame has a bytelength twice the number of pixels with the same number of rows. In anembodiment, the output clock rate is increased, for example, from 10 MHzto 40.5 MHz, to maintain the original refresh rate for the display andraw data. Although a 320×240 sensor is illustrated, embodiments of thepresent invention are not limited to this particular spatial resolutionand other resolutions, including 640×480 as discussed in relation toFIG. 4 can be utilized. In some embodiments, the frame rate ismaintained at the original frame rate of the camera but delivers moredata than a normal frame, for example, twice the data when the superframe has twice the data content of a normal frame.

The framer performs an interleaving function. The 14-bit data isillustrated as two bytes of the following format: PxBy, where x=thepixel number and y=the byte (either 0 or 1).

For example:

-   -   P0B0=Pixel 0, byte 0=bits 7-0,    -   P0B1=Pixel 0, byte 1=bits 15-8 (with bits 13-8 as valid data and        bits 14 and 15 set to zero)

The data format illustrated in Table 1 is utilized for each row of thesuper frame as illustrated in FIG. 3. The framer includes a buffer thatholds one line of data from the colorizer module and one line of datafor the raw data and frames the data to be output on a parallel 8-bitbus (i.e., one byte per clock cycle). In other embodiments, a 16-bit busis used with two bytes transmitted per clock cycle. As shown in FIG. 3,each row of the colorized data is 640 bytes since there are 320 pixelsand each pixel uses two bytes. The 16 bits of the raw video data isbroken up into an upper byte and a lower byte for each of the 320pixels, resulting in 640 bytes of raw video data. For 14-bit data, twoof the bits are set to zero to form the 16-bit video data. Accordingly,the framer interleaves the 640 bytes of color data and the 640 bytes ofraw video data to form the super frame. After framing, each row ofcolorized data is transmitted followed by the raw data in the same row.

FIG. 4 is diagram illustrating a frame format for 640 pixel wide videooutput according to an embodiment of the present invention. For thishigher pixel resolution sensor, the 640×480 pixels result in 480 rowswith 1280 bytes per row.

FIG. 5 is a diagram illustrating a super frame according to anembodiment of the present invention. As illustrated in FIG. 5, the superframe 510 includes both raw data 514 (e.g., 14-bit video) and aprocessed version 512 (i.e., processed data) of the raw data (e.g.,8-bit contrast enhanced video) that is suitable for imaging displays ina single super frame 510. The raw data 514 and the processed data 512are output using a single parallel digital output. The processed data isaligned to the raw data to enable the user to switch between the raw andprocessed data without the need for hardware switches.

As illustrated in FIG. 5, the super frame output stream includes both8-bit contrast enhanced video 512 and 14-bit video 514. The contrastenhanced video is very viewable, high contrast video, which isappropriate for display to a user. The 14-bit representation of the samescene illustrated by the raw data 514 appears washed out, but is usefulin analytics modules and video processing since it is characterized by ahigher bit depth resolution. By providing both the high bit depth imagesfor analytics and the lower bit depth, contrast enhanced images fordisplay in a single super frame, the present invention provides benefitsnot available using conventional techniques.

FIG. 6A is a simplified flowchart illustrating a method of providing adual video stream according to an embodiment of the present invention.The method 600 includes capturing a scene of imaging data using a videocamera (610). The imaging data is characterized by a first bit depth,for example 14-bits. Capturing the scene of imaging data can includeperforming at least one of temperature compensation or non-uniformitycorrection in order to provide imaging data that compensates fornon-uniformities across the detector, for example, a focal plane array,used to capture the imaging data. The method also includes processingthe imaging data to provide display data characterized by a second bitdepth less than the first bit depth (612). Processing the imaging datato provide display data can include performing gray scale conversion,performing image contrast enhancement, converting gray scale data tocolor data, or the like. As an example, 14-bit data can be converted to8-bit data as the bit depth is reduced from the first bit depth to thesecond bit depth.

The method also includes framing the imaging data and the display data(614) and outputting the framed imaging and display data (616). In someembodiments, the imaging data, the display data, and the framed imagingand display data are provided to a multiplexer that is able to selectone of the inputs as an output that is provided to a parallel interface.Accordingly, the imaging data (i.e., the raw data from the FPA) can beprovided as an output of the system.

It should be appreciated that the specific steps illustrated in FIG. 6Aprovide a particular method of providing a dual video stream accordingto an embodiment of the present invention. Other sequences of steps mayalso be performed according to alternative embodiments. For example,alternative embodiments of the present invention may perform the stepsoutlined above in a different order. Moreover, the individual stepsillustrated in FIG. 6A may include multiple sub-steps that may beperformed in various sequences as appropriate to the individual step.Furthermore, additional steps may be added or removed depending on theparticular applications. One of ordinary skill in the art wouldrecognize many variations, modifications, and alternatives.

FIG. 6B is a simplified flowchart illustrating a method of providing adual video stream using radiometry data according to an embodiment ofthe present invention. The method 645 includes capturing a scene ofimaging data using a thermal imager (650). The imaging data ischaracterized by a first bit depth, for example 14-bits. As part of theprocess of capturing the scene of imaging data, temperature compensationor non-uniformity correction can be performed as described above. Themethod also includes processing the imaging data to provide display datacharacterized by a second bit depth less than the first bit depth (652).As an example, the imaging data can be converted from 14-bits to 8-bitsin an embodiment. Processing the imaging data to provide display datacan include converting gray scale data to color data for display to auser, performing image contrast enhancement, and the like.

The method further includes processing the imaging data to provideradiometric data characterized by a third bit depth greater than thefirst bit depth (654). In some embodiments, the 14-bit intensity data isconverted to 16-bit radiometric data providing information on thetemperature of a pixel rather than the intensity measured for the pixel.As an example, a lookup table can be used in performing the radiometricconversion from intensity data to temperature data. Additionally, themethod includes framing the radiometric data and the display data (656)and outputting the framed radiometric and display data (658). Inaddition to the framed data, the imaging data and the display data canbe provided to a multiplexer that can be used to select the desiredoutput.

It should be appreciated that the specific steps illustrated in FIG. 6Bprovide a particular method of providing a dual video stream usingradiometric data according to an embodiment of the present invention.Other sequences of steps may also be performed according to alternativeembodiments. For example, alternative embodiments of the presentinvention may perform the steps outlined above in a different order.Moreover, the individual steps illustrated in FIG. 6B may includemultiple sub-steps that may be performed in various sequences asappropriate to the individual step. Furthermore, additional steps may beadded or removed depending on the particular applications. One ofordinary skill in the art would recognize many variations,modifications, and alternatives.

FIG. 11 is a simplified block diagram illustrating a thermal imagingsystem according to an embodiment of the present invention. Asillustrated in FIG. 11, the thermal imaging system 1100 includes one ormore optical elements in optics module 1110 that collects infrared lightfor use in the thermal imaging system. The camera core 1120 is opticallycoupled to the one or more optical elements and receives the infraredlight transmitted through the optics module 1110. As illustrated in FIG.1, the camera core includes an FPA module providing imaging data at afirst bit depth, a color conversion module coupled to the FPA module andproviding display data at a second bit depth less than the first bitdepth, and a framer coupled to the FPA module and the color conversionmodule. The framer creates a super frame that includes the imaging dataand the display data.

The thermal imaging system also includes a communications module 1130that is coupled to the camera core. The communications module isoperable to interact with network 1160, which can be used to receivethermal imagery from the system, provide control inputs for the system,and the like. Memory 1140 is provided that is able to store data fromthe camera core, store settings and calibration data used by the cameracore, and the like.

The thermal imaging system also includes an input/output module 1150 inthe embodiment illustrated in FIG. 11 that is a coupled to thecommunications module. The I/O module enables a system operator or userto interact with the system to provide command and controlfunctionality. In some embodiments, the I/O module is optional. In otherembodiments, the I/O module and the communications module are combinedinto a single module that provides some or all of the functionsdescribed for these separate modules. One of ordinary skill in the artwould recognize many variations, modifications, and alternatives.

Embodiments of the present invention can utilize an LVDS interface thatsupports two or more modes of operation: Camera Link® mode and YUV Superframe mode as a serialized version of the parallel output discussedabove. The Camera Link® mode is typically used to interface to CameraLink® frame grabbers. The LVDS video interface supports 4 LVDS datapairs and the LVDS clock pair as outputs. The LVDS timing is shown inTable 2, while the timing diagram is shown in FIGS. 7 and 8. The LVDSClock signal has a non-fifty percent duty cycle. It is based on a 7×internal clock. The LVDS Clock is high for 4 of the 7× clock periods andlow for 3. During each clock period, 7 bits are transmitted on each datapair. The bits are transmitted in the order shown in FIG. 7 with eachpixel value starting in the middle of the high clock period. The LVDSdata window timing is shown in FIG. 8. The maximum delay for the data tobecome valid after clock and the minimum time data will go invalidbefore the clock are also described in Table 2, which illustrates theLVDS timing and framing.

TABLE 2 Number Parameter Min Nom Max Units 1 Clock Period 48.6 ns 7xInternal Clock 144 MHz Freq. Bit time 5.94 ns 2 Data no longer valid 0.4ns before clock 3 Clock to data valid 0.4 ns 4 Data valid window 6.14 ns#LINES Lines per frame 480 #PIXELS_CL Pixels per line in 640 Camera LinkMode #PIXELS_YUV Pixels per line in 1280 YUV-SF Mode

FIG. 9 shows the mapping of a Camera Link® serialized bit stream to a 24bit RGB color bit stream. FVAL is low (invalid) between frames whileLVAL is low (invalid) between lines. DVAL is high to indicate that thedata is valid. A frame will consist of FVAL going high (valid) for anentire frame.

Blanking time is inserted between each frame while FVAL is low. A linewill consist of LVAL going high (valid) for an entire line. Blankingtime is inserted between each line while LVAL is low. The amount ofhorizontal and vertical blanking can change based on operating modes andCamera revisions.

The LVDS Interface supports three interface formats in the embodimentillustrated in FIG. 9:

-   -   1. 14-bit/8-bit Gray Scale    -   2. 24-bit RGB    -   3. YUV Super frame

The 14-bit Gray Scale format is used to support the 14-bit and 8-bitgray scale data modes. The 14-bit and 8-bit Gray Scale mapping followsthe Camera Link® standard and maps as shown in Table 3. The 24-bit RGBformat is used to support the colorization data mode and uses thestandard Camera Link® 24-bit RGB format. The 24-bit RGB format can beutilized as an alternative implementation compared to the 4:2:2 colormode discussed previously. As will be evident to one of skill in theart, the 4:2:2 color mode uses 16 bits per pixel, which is less than the24 bits per pixel used in the 24-bit RGB format. Accordingly, the 4:2:2color mode can be utilized in place of the 24-bit RGB format. Thus, thesuper frame can be sent through the LVDS Camera Link® interface usingthe bit mapping illustrated in Table 3.

TABLE 3 Camera Link ® 24- 14-bit data 8-bit data YUV Super frame Bitcolor Mode Mode Mode Mode G7 Not Used Not Used Bit 15 G6 Not Used NotUsed Bit 14 G5 Bit 13 Bit 7 Bit 13 G4 Bit 12 Bit 6 Bit 12 G3 Bit 11 Bit5 Bit 11 G2 Bit 10 Bit 4 Bit 10 G1 Bit 9 Bit 3 Bit 9 G0 Bit 8 Bit 2 Bit8 R7 Bit 7 Bit 1 Bit 7 R6 Bit 6 Bit 0 Bit 6 R5 Bit 5 Not Used Bit 5 R4Bit 4 Not Used Bit 4 R3 Bit 3 Not Used Bit 3 R2 Bit 2 Not Used Bit 2 R1Bit 1 Not Used Bit 1 R0 Bit 0 Not Used Bit 0

In YUV Super frame mode, a 16-bit video stream is mapped into the CameraLink® Interface as shown in Table 3. The YUV Super frame consists of 480lines with each line containing 1280 values. The first 640 valuescontain YCbCr generated values for the pixels of that line with thesecond 640 values containing the pre-AGC values for that line (currentlythe pre-AGC values are from the frame before the current YCbCr frame,this allows time for analytics to analyze the pre-AGC data so additionaloverlays can be added to the YCbCr data stream by customer analytics).

FIG. 10 illustrates a YUV Super frame line format for one line of dataaccording to an embodiment of the present invention. The first Cb and Crdata is generated on the average of the first two pixels. The second Cband Cr data is generated on pixels 3 and 4 with all further Cb/Cr pairscalculated in a relative manner. The Pre-AGC data is LSB aligned so ifthe Pre-AGC data is only 14-bits, it will only occupy the lower 14 bitsof the data path respectively.

Table 4 illustrates timing for several modes of operation according toan embodiment of the present invention. The modes of operation areassociated with the four inputs provided to the multiplexer 140 in FIGS.1 and 2. As discussed above, although not illustrated in FIG. 1 or 2,the super frame could be formed using the 8-bit AGC and the raw data(FIG. 1) or the 8-bit AGC and the 16-bit radiometric data (FIG. 2). Oneof ordinary skill in the art would recognize many variations,modifications, and alternatives.

TABLE 4 Pixel clocks Frame Pixel Effective Camera Mode of per data RateClock Frame Rate Type Operation sample (Hz) (MHz) (Mbps) Tamarisk 8-bitAGC 1 60 5 37 320 4:2:2 Color 1 60 5 74 14-bit Raw 1 60 5 74 4:2:2/Raw 430 24 74 Super Frame Tamarisk 8-bit AGC 1 30 10 74 640 4:2:2 Color 1 3010 148 14-bit Raw 1 30 10 148 4:2:2/Raw 4 15 24 148 Super Frame

It is also understood that the examples and embodiments described hereinare for illustrative purposes only and that various modifications orchanges in light thereof will be suggested to persons skilled in the artand are to be included within the spirit and purview of this applicationand scope of the appended claims.

What is claimed is:
 1. A thermal imaging system comprising: one or moreoptical elements operable to collect infrared light; a camera coreoptically coupled to the one or more optical elements, wherein thecamera core includes: a focal plane array (FPA) module providing imagingdata at a first bit depth; an image processing module coupled to the FPAmodule and operable to process the imaging data to provide display data;a framing module to form a super frame including the image data and thedisplay data; a communications module coupled to the camera core; and aninput/output module coupled to the communications module.
 2. The thermalimaging system of claim 1 wherein the display data comprises YUV displaydata and the imaging data comprises pre-automatic gain control (pre-AGC)image data.
 3. The thermal imaging system of claim 2 wherein the YUVdisplay data is formatted for a set of pixels as luminance and a Cbvalue for a first pixel and luminance and a Cr value for a second pixel.4. The thermal imaging system of claim 3 the Cb value is an average forthe first pixel and the second pixel and the Cr value is an average forthe first pixel and the second pixel.
 5. The thermal imaging system ofclaim 1 wherein the super frame comprises 480 lines, each of the 480lines having 1280 values comprising: a first set of 640 values, whereinthe first set of 640 values contain YCbCr generated values for a set ofpixels; and a second set of 640 values, wherein the second set of 640values contain pre-AGC values.
 6. The thermal imaging system of claim 1wherein the display data is characterized by a second bit depth lessthan the first bit depth.